Systems And Methods For Analyzing Event Data

ABSTRACT

A computer-implemented method for determining a target situation in an athletic event. Positional information including the relative positions of a group of selected participants is initially received from a tracking system, and the aggregate motion of the selected participants is detected in real-time using the positional information. The target situation may be determined to have occurred when a change in the aggregate motion occurs in accordance with a predetermined characteristic during an initial time interval.

RELATED APPLICATIONS

This application claims priority to U.S. Patent Application Ser. No.61/292,386, filed Jan. 5, 2010, the disclosure of which is incorporatedherein by reference.

BACKGROUND

Systems that track objects in an event, such as participants in anathletic event, are known. For example, U.S. Patent ApplicationPublication No. 2008/0129825 to DeAngelis et al., which is incorporatedherein by reference, discloses systems and methods to facilitateautonomous image capture and picture production. A location unit isattached to each tracked object (e.g., participants in an athleticevent). An object tracking device receives location information fromeach location unit. A camera control device controls, based upon thelocation information, at least one motorized camera to capture imagedata of at least one tracked object.

It is also known to manually create video and still images of an event.For example, a video feed of an event (e.g., an American football game)is typically generated by highly trained camera persons and highlytrained production staff who select camera shots and combine graphicsinto the video feed. Video images and/or still picture production can bepartially or fully automated using systems and methods disclosed in U.S.Patent Application Publication No. 2008/0129825.

In many American football games, two ‘standard’ views are manuallyfilmed using two digital video cameras; one on the sideline, and one inan end zone. These views are then manually ‘broken down’ by humanswatching the videos, clipping them into plays, and identifyinginteresting attributes of each play. One of the most obvious attributesis simply who was on the field for each team at a given time. This isalso one of the most difficult things to determine from the video sincethe resolution is not sufficient to clearly determine each of theplayers' numbers, thus making it difficult to identify all of theplayers.

SUMMARY

A computer-implemented method is disclosed for determining a targetsituation in an athletic event. In one embodiment, positionalinformation including the relative positions of a group of selectedparticipants is initially received from a participant tracking system.Aggregate motion of the selected participants is detected in real-timeusing the positional information. The target situation is determined tohave occurred when a change in the aggregate motion occurs in accordancewith a predetermined characteristic during an initial time interval.

In another embodiment, a video feed of an event is annotated byreceiving positional information indicating the position of a selectedparticipant in the event from a tracking system. The path of travel ofthe participant is determined from the positional information, andgraphical information indicating the path of travel, and informationidentifying the participant, is overlaid onto the video feed.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 shows an exemplary system for automatically generating eventcharacterization information from event data;

FIG. 2 shows the system of FIG. 1 in greater detail;

FIG. 3 shows an example in which aggregate motion of event participantsis used to determine the occurrence of a particular target situation inan event;

FIG. 4A shows an exemplary method that is used with certain embodimentsto detect a target situation in an event by analyzing the aggregatemotion of participants in the event;

FIG. 4B is a flowchart showing exemplary steps performed in detecting aplay in a sporting event;

FIG. 5A is a flowchart showing exemplary steps performed in usingpositional information to determine certain target situations;

FIG. 5B is a flowchart showing exemplary steps performed in generatingsolutions and recommendations to increase a team's future successpercentage;

FIG. 5C is an exemplary diagram showing a standard line formation and adeviant of the standard formation;

FIG. 5D is a flowchart showing exemplary steps performed in predictingan opponent's behavior based on detecting static formation deviationswhich are correlated with historical behavior;

FIG. 5E is a flowchart showing exemplary steps performed in generatingsolutions and recommendations to improve the performance of a team basedon detecting deviations in dynamic play execution

FIG. 5F is an exemplary diagram showing a standard route and a deviantroute in two similar patterns (paths) run by a wide receiver;

FIG. 5G is a flowchart showing exemplary steps performed in predictingan opponent's behavior based on detecting deviations in dynamic playexecution and correlating these deviations to previous behavior inspecific situations;

FIG. 5H is an exemplary diagram showing a standard route and a deviantroute in two similar patterns (paths) run by a slot receiver;

FIG. 6A is a flowchart showing exemplary steps performed in usingpositional information to provide automatic annotation of a video feedof an event;

FIG. 6B is an exemplary diagram showing player identification graphicsindicating a video display;

FIG. 6C is an exemplary diagram showing player identification graphicsindicating the off-screen location of an object;

FIG. 6D is an exemplary diagram showing a window containing ahighlighting shape overlaid on a video stream;

FIG. 7 is a flowchart showing exemplary steps performed in evaluating aparticipant's performance;

FIG. 8 is a flowchart showing exemplary steps performed in automatingthe video filming of selected parts of an entire game; and

FIG. 9 is a diagram showing the use of a wireless device with thepresent system.

DETAILED DESCRIPTION

The present disclosure may be understood by reference to the followingdetailed description taken in conjunction with the drawings describedbelow. It is noted that, for purposes of illustrative clarity, certainelements in the drawings may not be drawn to scale.

Systems and methods disclosed herein analyze event data, such asrespective real-time locations of participants in an event, toadvantageously automatically generate information characterizing one ormore aspects of the event (event characterization information). In thecase of a sporting event, examples of possible event characterizationinformation include (1) identification of formations and/or plays, (2)beginning and/or end of a play, (3) players' paths of travel, (4) linesof scrimmage, (5) identification of players, and (6) position andorientation of coaches and/or officials.

Such event characterization information, for example, is provided tointerested recipients (e.g., event spectators, coaches, and/or eventofficials) automatically or upon demand. In some embodiments, eventcharacterization information is advantageously used to enhance a videofeed 109 of an event, such as by overlaying graphical information ontothe video feed. Some embodiments of the systems and methods disclosedherein may partially or fully automate imaging of an event (e.g.,generating a video feed of the event) and/or control delivery of eventimages to recipients.

The present system and methods provide the basic functionality forimplementing, among other things, a real-time feed showing who is on aplaying field at all times. The feed can be automatically added as adata track to the original digital video.

FIG. 1 shows an exemplary system 100 for automatically generating eventcharacterization information from event data. System 100 receives avideo feed (live video stream) 109 and other event data 102, whichincludes information such as participant location information and systeminstructions 107, and automatically generates event characterizationinformation 104. Exemplary embodiments of system 100 are operable toautomatically generate event characterization information 104 inreal-time, where the term “real-time” in the context of this disclosureand appended claims means information 104 is generated as the eventoccurs. For example, identification of a play in a sporting event inreal-time means that the play is identified as it occurs, as opposed tothe play being identified at a later time (e.g., upon post-analysis ofthe event).

FIG. 2 shows exemplary system 100 in more detail. System 100 includes aninput/output (I/O) subsystem 106 operable to receive event data 102 anduser input 118, and to output event characterization information 104.I/O subsystem 106 includes, for example, a USB (Universal Serial Bus)and/or Ethernet interface for connecting to one or more externalsystems. It is to be noted that input 118 is received from a system userto initiate and/or provide certain information specific to each of thesystem functions described below.

In one embodiment, I/O subsystem 106 is communicatively coupled to avideo feed 109 from one or more video cameras 117, and a tracking system108, information from which is transmitted via link 107, which alsoprovides data including information pertaining to event participants andinstructions for system 100 including requests to be processed. Videocameras 117 may be manually or robotically controlled. Tracking system108 determines positions and/or velocities of event participants fromlocation units such as active RFID tags affixed to event participants bytriangulating the position of the location units to determine respectivepositions and/or velocities of the participants. System 100 mayalternatively receive event data 102 from Internet 110 via I/O subsystem106.

In the present embodiment, system 100 further includes a processor 112,a data store 114, and a database 115. Processor 112 is a computingdevice, which includes, for example, a general purpose microprocessor,processes event data 102 to generate event characterization information104 in response to instructions 116, in the form of software and/orfirmware 116, stored in data store 114. Examples of methods executed byprocessor 112 to generate event characterization information 104 arediscussed below.

Data store 114 typically includes volatile memory (e.g., dynamic randomaccess memory) and one or more hard drives. Although the components ofsystem 100 are shown grouped together, such components could be spreadover a number of systems, such as in a distributed computing environmentand/or in a distributed storage environment.

In one embodiment, event characterization information 104 is transmittedto an annotation system 124, which annotates a video feed 120 (e.g., alive video feed) of the event to produce an annotated video feed 122. Incertain embodiments, annotation system 124 overlays graphicalinformation onto video feed 120, as is known in the art, and thegraphical information includes event characterization information 104.

The present system advantageously uses information derived from theaggregate motion of event participants. The aggregate motion of multipleparticipants in an event can indicate the occurrence of a particularincident, target situation, or circumstance of interest (hereinaftercollectively, “target situation”) in the event (e.g., the beginning of aplay in a sporting event). An aggregate motion value representscollective motion of two or more participants. An aggregate motion valuefor selected event participants at a given point in time can bedetermined, for example, by summing the velocities of the participantsat that time or determining an average velocity of the participants atthat time. A particular target situation can be detected by recognizingchanges in aggregate motion values and/or sequences of aggregate motionvalues that are known to coincide with the target situation, therebyindicating that the target situation occurred.

FIG. 3 shows an example in which aggregate motion of event participantsand knowledge of the type of event can be used to determine theoccurrence of a particular target situation in the event. Morespecifically, FIG. 3 is a graph 300 of normalized aggregate motionversus time for a 24 second interval of an American football game.Certain target situations in a football game can be recognized bycomparing actual aggregate motion values of graph 300 to aggregatemotion values known to occur with a given target situation. For example,segments A, B, D, and E are respectively characterized by a moderatevalue of aggregate motion, a small value of aggregate motion, a largeincrease in aggregate motion over a defined time duration, and asustained large value of aggregate motion. Such sequences of aggregatemotion values are known to occur with the preparation for and executingof an offensive play, and the beginning of an offensive play can thus beinferred from the sequence.

Specifically, segment A represents the offensive squad breaking a huddleand heading toward a line of scrimmage, segment B represents anoffensive squad assembling at a line of scrimmage and entering apre-play condition, segment D represents a beginning of a play, andsegment E represents the play in progress. Accordingly, an offensiveplay can be detected by recognizing the sequence of aggregate motionvalues associated with segments A, B, D, and E. Point C, which ischaracterized by a brief spike in aggregate motion, represents a playergoing into motion before the ball was snapped.

A sequence of aggregate motion values in graph 300 can also berecognized to determine an end of the play. In particular, segments E,F, G, and H are respectively characterized by a sustained large value ofaggregate motion, a substantial decrease in aggregate motion over adefined time duration, a moderate value of aggregate motion, andmoderate but smaller value of aggregate motions. Such sequence ofaggregate motion values are known to occur with the ending of a play. Inparticular, segment E represents the play in progress (as previouslynoted), segment F represents the end of the play, segment G representsthe players moving from the post play positions back to the next huddle,and segment H represents players beginning to assemble for the nexthuddle.

Accordingly, certain embodiments of system 100 at least partiallydetermine event characterization information 104 from aggregate motionof participants of the event, such as by processor 112 executinginstructions 116 to perform an analysis similar to that discussed abovewith respect to FIG. 3.

FIG. 4A shows an exemplary method 400 that can be used with certainembodiments of system 100 to detect a target situation in an event byanalyzing the aggregate motion of participants in the event inreal-time. Method 400 is performed, for example, by processor 112 ofsystem 100 executing instructions 116. As shown in FIG. 4A, in step 402,aggregate motion of a number of participants of the event is detected,such as by processor 112 calculating an average velocity of selectedparticipants in the event from velocity data calculated from event data102. Processor 112 may calculate respective participant velocities fromchanges in participant positions and then determine an average velocityfrom the respective participant velocities. Calculated aggregate motionvalues may be stored in database 115 for subsequent use.

In step 404, a change in aggregate motion is determined. For example,processor 112 may determine a difference between two sequentiallydetermined aggregate motion values stored in database 115. In step 406 atarget situation is detected if the change in aggregate motion detectedin step 404 has a predetermined characteristic (block 407), and/or if aspecific sequence of aggregate motion values detected (block 408),and/or if specific positions or orientation of event participants isdetected (block 409).

For example, with respect to block 407, processor 112 may detect thebeginning of a sporting event play if the change in aggregate motionmeets a predetermined increases by at least a threshold value within agiven time period as specified by instructions 116, such as similar tothe increase shown in segment D of graph 300 (FIG. 3). As anotherexample of step 406, processor 112 may detect an end of the sportingevent play if aggregate motion decreases by at least a threshold valuewithin a given time period as specified by instructions 116, such as avalue similar to the decrease that occurred in segment F of graph 300.

In block 408, a specific sequence of aggregate motion values must occurbefore a target situation is determined to be detected. For example,detection of a play beginning may require a minimum aggregate motionvalue to precede a rapid increase in aggregate motion values and/or amaximum sustained aggregate motion value to follow the rapid increase,similar to sequences B, D and D, E of FIG. 3, respectively. As anotherexample, detection of a play ending may require a maximum sustainedaggregate motion value to precede a rapid decrease in aggregate motionvalues and/or a moderate aggregate motion value to following the rapiddecrease, similar to sequences E, F, and F, G of FIG. 3, respectively.

In block 409, event data 102 must have certain characteristics inaddition to known aggregate motion characteristics to detect a targetsituation. Examples of such additional characteristics include positionsand/or orientation of participants relative to each other or relative toa playing field. In the case of a football game, the beginning of a playmay be detected if a certain number of players are in certainpredetermined positions indicating formation of a starting line prior toa sufficiently rapid increase (e.g., 6 feet/second minimum aggregatespeed in a 0.3 second period) in aggregate motion values.

The choice of the specific participants to be considered whendetermining the aggregate motion value in step 402 depends on thespecific intended application of method 400. For example, in the case ofAmerican football, only the players on one team might be considered whendetermining an aggregate motion value. As another example, only playersconsidered likely to be involved in a high level of motion during aparticular target situation, such as running backs, receivers, andquarterbacks, may be considered when determining an aggregate motionvalue. The specific participants considered when determining anaggregate motion value may vary depending on the target situation to bedetected or determined. For example, different players may be consideredin aggregate motion determinations when detected in the beginning of anoffensive play and the kicking of a field goal.

FIG. 4B is a flowchart showing exemplary steps performed in detecting aplay in a sporting event. As shown in FIG. 4B, initially, at step 410,specific players in a sporting event are selected for inclusion in anaggregate motion tabulation. Combining the motion of multiple playersminimizes the impact of the random movement of individual players andaccentuates the differential movement associated with specific targetsituations. Certain players or players at certain positions inherentlyexhibit higher levels of differential motion than others. Selectingplayers with typically high levels of differential movement for theaggregate tabulation, and ignoring the remaining players, minimizes theeffect of random motion while maximizing differential motion levels atvarious stages of a target situation.

In an American football game, certain ‘skill’ positions have arelatively high level of differential motion associated with thebeginning or end of a play, thus their inclusion in an aggregate motiontabulation increases the differential levels of aggregate motion. Skillpositions include wide receivers, running backs, and defensive backs.Linemen typically have low differential motion during play start/stopand so their inclusion in the tabulation reduces the differential levelsof aggregate motion.

At step 412, a pre-play target set of conditions is identified.Situations of interest are generally preceded by a definable(sport-specific) set of conditions, players, positions, relativemovements, and the like. The occurrence of this target set of conditionsis an indication that a target situation will occur in the near futureand is used as a pre-condition to a refined set of player position andalignment criteria.

The pre-play target conditions in American football are met when thereare exactly 11 players from each team on the playing field and bothteams are on their own side of the line of scrimmage. This situationoccurs toward the end of segment A in the graph shown in FIG. 3.

At step 414, a system ‘arm’ condition is identified. In addition topre-play conditions, a target situation is often immediately preceded bya definable (sport-specific) set of conditions, players, positions,relative movements, and the like. A system arm condition is anindication that the target situation is imminent and is used as apre-condition to more specific, motion based criteria, described below.

In American football one arm condition is known as a ‘line set’. Thiscondition is defined by a certain number of linemen being stationary fora defined period (typically <800 ms) and the offensive and defensivelinemen being positioned within a defined distance of each other(typically <2 meters). This situation occurs toward the end of segment Bin the graph shown in FIG. 3.

At step 416, a start-of-play condition is identified. The beginning of atarget situation (e.g., start of play) is characterized by a specificaggregate motion profile. In most cases this will be a rapid increase inaggregate motion but depending on the sport other aggregate motionprofiles may exist. If the real time aggregate motion profile matchesthe aggregate motion start profile then the start of a situation hasbeen detected.

In American football, immediately prior to the snap of the ball, alloffensive players (with minor exceptions) are required to be motionless.This condition results in a very low aggregate motion baseline, whichwas established during the arm condition. As soon as the ball issnapped, all players begin moving nearly simultaneously, with theposition players often moving rapidly. This results in the aggregatemotion radically increasing over a very short period of time. Thissituation matches the profile for start-of-play and occurs toward theend of segment D in the graph shown in FIG. 3.

At step 418, an aggregate motion baseline of play is established.Following a start event the target situation will typically reach andmaintain some level of sustained aggregate motion. This establishes anaggregate motion baseline value for the play.

Following the start of a play in American football, the players aretypically moving at a reasonably stable level of aggregate motion. Themagnitude of this level will vary depending on the type of play. In thecase of a long pass play, the level of aggregate motion will berelatively high, and on a running play it will be relatively low.Regardless of the type of play, a sustained aggregate motion of somelevel will generally be established. This condition exists as segment Ein the graph shown in FIG. 3.

At step 420, an end-of-play condition is identified. The end of thistarget situation (i.e., end-of-play) is characterized by a specificaggregate motion profile. In most cases this profile exhibits a gradual,yet constant, decrease in aggregate motion, with an initially fasterdecrease (e.g., a decrease in aggregate speed of 40% in 0.5 seconds) inthe motion. If a particular real time aggregate motion profile matchesthe aggregate motion stop profile then the end of a play has beendetected.

In American football, when the referee blows his whistle, indicatingthat a play has ended, the players will begin to slow down. While theaggregate motion will immediately begin to decline, since the players donot all stop instantaneously, or at the same instant, the decline willbe more gradual than the play start. However, the end-of-play profile isidentified by aggregate motion consistently decreasing over a predefinedrelatively short period of time, for example, 800 milliseconds. Inpractice, this duration is dictated by the specific sport and thespecific situation of interest in that sport. This condition exists assegment F in the graph shown in FIG. 3.

Once a target situation has ended, system 100 begins looking for thenext target situation to enter its pre-play condition. In an Americanfootball game, once a play is over the system monitors the players,positions, etc., as described above, seeking to identify the nextpre-play condition. This condition exists in segment G and carriesthrough into segment H in the graph shown in FIG. 3. If a game is stillin progress at this point, the procedure described above resumes at step412.

As noted above, the present system can determine the occurrence of atarget situation at least partially from positional information.Examples of target situations determined from analysis of positionalinformation include players breaking (from) a huddle, reaching a setposition in a line formation, and the beginning of a play. For example,in an American football game, players' positions relative to each other,or relative to a particular yard marker may indicate that the playersare lined up at the line of scrimmage immediately prior to beginning aplay.

FIG. 5A is a flowchart showing exemplary steps performed in usingpositional information to determine certain target situations inreal-time. As shown in FIG. 5, at step 505, tracking information, whichincludes the relative positions of event participants of interest, isreceived from tracking system 108. At step 510, the relative positionsof selected participants (e.g., players on a particular team) aredetermined from analysis of the tracking information. At step 515, ifthe positions of the selected participants meet certain predefinedcriteria, then a corresponding target situation is detected, at step520.

In one embodiment, the predefined criteria includes relative positionsof participants determined by analyzing the tracking information todetect the participants' positions relative to certain positionindicators, such as yard line markers in a football game. The criteriamay also include the orientation of participants. e.g., the direction inwhich the participants are facing.

Examples of target situations that can be determined from positionalinformation include team in huddle, players in a particular formation,and players' position relative to the line of scrimmage. Relativepositions of coaches and officials can enable detection of a targetsituation such as a coach signaling a ‘time out’ or an officialsignaling a penalty. Positional information may also be used to analyzeactions of officials and assist with their training.

In American football, there are a finite number of basic offensive anddefensive formations in common use. On top of these there are manystandard variations of these formations. Some of these variations arecommon to nearly all teams, while others are specific to individualteams. Beyond these standard and variant formations there are aninfinite number of subtle formation variations, both intentional andunintentional.

Currently, defensive coaching staffs routinely study an upcomingopposition's offensive formations and subsequent playselection/execution. In conjunction with various game situations (e.g.,3rd down and long) they calculate the percentage of time an opponentruns certain plays in specific situations. For example, in a ‘third andlong’ situation, when in a particular formation, the offense passes theball 75 percent of the time and when passing from this formation theball is passed it to a wide receiver 43 percent of the time.

The objective of compiling these statistics is to improve the accuracywith which the defense can predict which play the opposing offense willrun in a given situation and, in turn, select the defensive formationwith the highest likelihood of success. The identification of subtlevariations in player formations allows the systematic prediction ofwhich play the offense is most likely to run. An offense mayintentionally employ a subtle formation variation as they believe thereis advantage to be gained from this variation based on the play that isabout to be run. For instance their pass blocking may be more effectivewhen employing a very subtle increase in lineman spacing.

In analyzing video data from a team's past performances this variationcan be systematically identified. This analysis may lead to learningthat, in a third and long situation, when in a particular formation, andwhere the offensive line assumes a slightly wider space than normal, aparticular offense passes the ball a certain percentage (e.g., 95.8percent) of the time.

The present system compares formations, on a play by play basis, againsta catalog of historical plays of the same class and systematicallyidentifies subtle formation variations within each specific play. In themethods shown in FIGS. 5B-5H (described below), a computer program 116is used to systematically determine statistically significantcorrelations between subtle formation variations and plays run whenthese specific subtle variations were present. Each of the examples inFIGS. 5B-5H is set forth in the context of American football;nevertheless, the methods described in accordance with these figures areapplicable to other sports as well. This process systematically distillsan infinite possible number of subtle variations down to a finite numberof meaningful predictors, which increases play prediction accuracy,improves the ability to choose the most appropriate formation and thusmay systematically improve a team's success percentage.

FIG. 5B is a flowchart showing exemplary steps performed in generatingsolutions and recommendations to increase a team's future successpercentage based on detecting deviations in static formations,correlating these deviations to specific outcomes (play results) andcomparing these correlations to the outcomes of previous situations.FIG. 5C is an exemplary diagram showing a standard line formation 553and a deviant 555 of the standard formation, wherein “X”s indicate theplayers on one team. Operation of the present system is best understoodby viewing FIGS. 5B and 5C in conjunction with one another.

Using player location data for a group of players (such as an offensivefootball squad in the present example), at a particular point in a game(i.e. just before a situation of interest, such as the snap of theball), the relative positions of the players is established, at step530, in FIG. 5B. Player location data can be acquired from trackingsystem 108 via feed 107. The relative positions of these players definea static formation 553 for that group of players, which formation isassociated with the subsequent play.

The static formation 555 established in step 530 is compared against alibrary (in database 115) of well known classes of standard formationsand accepted variants of those standard formations to identify a bestcase match with a standard formation, at step 532. In the example shownin FIG. 5C, the standard formation thus identified is shown in box 553.In this particular standard formation 553, the line spacing (distancebetween the players at the left and right tackle positions, as indicatedby marker 550) is 7 yards, and wide receiver X1 (circled) is lined up 5yards away from the right tackle, as indicated by marker 552.

Once a best case match has been made, deviations between the determinedstatic formation 555 and the standard library version of that formation553 are identified, at step 534. These deviations can be as subtle as,for example, where the average line spacing is slightly wider (8 yards,as indicated by marker 551) than in the standard library formation (7yards in the present example). These deviations may be significantlylarger, as where a wide receiver lines up 10 yards away from therespective tackle (as indicated by marker 553), as opposed to 5 yards(as indicated by marker 552), per the standard library formation shownin FIG. 5C.

Having identified a deviation between the previously captured staticformation 555 and the standard library formation 553, at step 536 thisdeviation is logged to database 115 along with a number of associatedattributes such as deviation type (e.g., wide offensive line spacing),matched library formation (class & variant), play results (success orfailure), and opposing formation (which type of defense was set upagainst the deviant offense in the static formation). Although yardagegained or lost is one measure of success, there may be other, moreappropriate, measures of success depending on the circumstances. Forexample, if an offense is facing third down and 10 (yards to a firstdown) and they gain 9.8 yards, then with respect to gain vs. loss, theplay might be judged, in the abstract, to be a success, but in thisparticular situation it was actually a failure. The above example isspecific to football and the parameters of success/failure will varywith specific situations.

The above examples represent only two deviations which might beidentified. In practice there may be ‘intentional’ deviations and manysubtle, ‘unintentional’ deviations from the standard formation. Althoughthe majority of these deviations may be tentatively deemed irrelevant tothe play outcome, all deviations are nevertheless logged into database115, as they may become relevant in the future as additional data iscollected.

Once a best case match has been made, deviations between the staticformation 555 and the standard library version 553 of that play aresystematically evaluated. At step 538, system 100 accesses playdeviation information in database 115 to identify deviations for whichthere are multiple instances and correlates these to play outcomes (bothpositive and negative).

Having identified these correlations, at step 540 these play outcomesare then compared to play outcomes when a particular deviation was notpresent, i.e., the deviant formation outcomes are compared against playoutcomes resulting from corresponding ‘standard’ formations. Previousformations, with associated deviations, are repetitively comparedagainst standard formations to get a best-case match for each, whichinformation is then logged in database 115 along with attributesindicating, such things as the success/failure of the formation (e.g.,the number of yards gained/lost using a particular deviant offensiveformation against a specific defensive formation).

At step 542, the system uses the correlations thus established togenerate a report for the coaching staff proposing solutions and/orrecommendations such as those indicated in the example below:

Positive Outcome Variation Detected

Squad: Offense

Formation Class: passing

Formation Variant: split wide receiver

Deviation Type: increased line spacing

Standard Success: 52.6%

Deviation Success: 63.1%

Recommendation(s):

-   -   Increase line spacing in split receiver formations.    -   Investigate line spacing increases in passing class formations.

FIG. 5D is a flowchart showing exemplary steps performed in predictingan opponent's behavior based on detecting deviations in staticformations and correlating these deviations to historical behavior ingiven situations. Operation of the present system is best understood byviewing FIGS. 5D and 5C (described above) in conjunction with oneanother.

Using player location data for an offensive football squad, in thepresent example, at a particular point in a game (i.e. just before asituation of interest, such as the snap of the ball), the relativepositions of those players is established, at step 580, in FIG. 5D. Therelative positions of these players define a static formation 555 (shownin FIG. 5C) for that group of players, which formation is associatedwith the subsequent play.

The static formation 555 established in step 580 is compared against alibrary (in database 115) of classes of standard formations for thespecified team of interest and accepted variants of those standardformations, for a specific team of interest, to identify a best casematch with a standard formation used by that team, at step 582. In theexample shown in FIG. 5E, the standard formation thus identified isshown in box 553. In this particular standard formation 553 (which isthe same formation as indicated in the example of FIG. 5B), the linespacing is 7 yards, and wide receiver X1 is lined up 5 yards away fromthe right tackle.

Once a best case match has been made, potentially significant deviationsbetween the defined static formation 555 and the standard libraryversion 553 of that formation are identified, at step 584. Havingidentified a deviation between the static formation 555 and the standardlibrary formation 553 for the team of interest, at step 585 thisparticular deviation is logged to database 115 along with a number ofassociated attributes such as deviation type (e.g., wide offensive linespacing), matched library formation (class & variant), situation (e.g.,which down and the number of yards to go), and subsequent type of playrun. This type of information may be used by a defensive squad toanalyze an offensive squad for specific ‘down and distance’ situationsto determine, on a statistical basis, what type of play this offensivesquad runs when faced with a particular situation, for example, thirddown and between 7 and 10 yards to a first down.

At step 586, system 100 accesses historical play data in database 115 toselectively retrieve previous plays for specific situational categories,for example, first down and ten yards to go, from between the opponent's10 and 20 yard lines, for a team of interest. At step 587, the resultsare then sorted into groups based on standard formations for the team ofinterest and a tabulation is made of the percentage of times specificplays were run from this standard formation given a specific type ofgame situation. The results are then further sorted based on common,identifiable, and sometimes subtle, deviations from the standardformation 553. After identifying correlations between formationdeviations and their outcomes, at step 588 these outcomes are thencompared to play outcomes when a particular deviation was not present,i.e., the deviant formation outcomes are compared against play outcomesresulting from corresponding ‘standard’ formations.

At step 589, a report is generated in which these tabulations arecataloged based on situations of interest for the coaching staff. Thereport is used in preparing a team to more accurately predict what theteam of interest will do in a given situation, from a specific formationand how specific deviations in that formation refine the likelihood of aparticular behavior. A typical report may include information such asthat indicated in the example below:

Behavior Prediction Based on Situation, Formation and Variant

Squad: offense

Down: third

Yardage: 7≦x≦10

Formation Class: passing

Pass 80%

Run 20%

Formation variant: split wide receiver

Deviation type: increased line spacing

Pass: 85%

Run: 15%

Formation Variant: split wide receiver

Deviation Type: increased wide receiver spacing

Pass: 93%

Run: 7%

Play recognition is a type of target situation that may be detected bythe use of information such as the path of travel of an eventparticipant, as determined from positional, velocity, and pathinformation. This information can be compared to a database of knownplays to recognize a particular type of play. In the embodimentsdescribed below with respect to FIGS. 5E-5H, database 115 is populatedwith information indicating previous formations and plays run by aparticular team in given game situations.

FIG. 5E is a flowchart showing exemplary steps performed in generatingsolutions and recommendations to improve the performance of a team basedon detecting deviations in dynamic play execution, correlating thesedeviations to specific outcomes and comparing the correlations to theoutcomes of previous situations. FIG. 5F is an exemplary diagram showinga standard route 573 and a deviant route 574 in similar patterns (paths)run by a wide receiver. Operation of the present system is bestunderstood by viewing FIGS. 5E and 5F in conjunction with one another.

Using a player location data set for a selected group of participants(such as an offensive football squad) captured for the full duration ofa situation of interest (e.g., an entire play), the path of eachindividual participant is determined, at step 590. The collection ofthese individual paths defines a dynamic play execution. In step 592,the dynamic play execution established in step 590 is compared against alibrary of well known classes of standard play executions (and acceptedvariants of those standard executions) stored in database 115, toestablish a best case match with a standard type of play.

This comparison is considered from the perspective of individual paths,which are compared to predefined paths and the paths treated as acollection of individual data points. Although there may be multiplepaths, each player has a predefined path, so the paths can be processedindividually. While the paths are actually two dimensional, they aretreated simply as collections of discrete data points, which can beevaluated for deviation from a standard path. What might be considered asignificant deviation will vary by sport, situation of interest, and byplayer position. When considering, for example, a wide receiver in anoffensive football play, a deviation of more than 1.5 yards frompredefined path may be considered significant.

In finding matches between deviations so that they can be groupedtogether, each standard play execution is considered as a collection ofindividual, predefined paths. Each individual path comprises acollection of specific segments consisting of legs and inflectionpoints. As an example, a wide receiver route might be described asfollows:

Wide Receiver Path Segments

Start of play

Segment 1—straight for 5 yards

Segment 2—90 degree turn toward center of field

Segment 3—10 yards straight

Segment 4—45 degree turn in opposite direction of segment 2 turn

Segment 5—straight until end of play

Once a path within a dynamic play execution has been identified, thenthe segment in which the deviation occurred is identified. Deviations inindividual paths are selected for further evaluation and, once selected,these paths are further classified such that they can be grouped with,and compared to, previously recorded deviations.

Once a best case match has been made between the dynamic play executionestablished in step 590 and a standard type of play, deviations betweeneach path of interest within the dynamic play execution set and thepaths defined in the standard library version of that play execution areevaluated, at step 594. In FIG. 5F, two paths for offensive player X1(e.g., a wide receiver) in formation 591 are shown—path 574 is the pathselected from the dynamic play execution established in step 590, andpath 573 is the path with the best case match selected from the standardlibrary of plays. The deviations determined by the evaluation made instep 594 may be as subtle as a wide receiver making a jog (at arrow 575)in his pattern where the receiver changes his ‘cut point’, as shown inFIG. 5F.

The present example represents one possible path deviation which mightbe identified. In practice there may be a large number of deviationspresent in a single play and possibly even multiple deviations in asingle player's path. Having identified a deviation between a pathwithin a dynamic play execution and the standard library path for thatplay execution, at step 595 this deviation is logged to database 115along with a number of associated attributes such as deviation type(e.g., wide receiver path), deviation specifics (e.g., additional coursechanges), matched library formation (class & variant), play outcome(success or failure), and opposing formation (which type of defense).Although the majority of the deviations may be tentatively deemedirrelevant to the play outcome, all deviations are nevertheless loggedin database 115 as they may become relevant in the future as additionaldata is collected.

At step 596, deviation information in database 115 is accessed toidentify significant correlations between various path deviations andplay outcomes (both positive and negative). Having identified thesecorrelations, at step 597 the outcomes are then compared tocorresponding standard play outcomes, that is, the results of a standardplay that had been executed as intended (e.g., as the play was initiallydrawn on a chalkboard) when a particular deviation was not present. Atstep 598, these correlations are then used to generate a report for thecoaching staff including relative success of deviant and standard paths,and optionally proposing solutions and recommendations. A typical reportmay include information such as the following:

Positive Outcome Variation Detected—Dynamic Play Execution

Squad: Offense

Execution Class Passing

Execution Variant Split wide receiver

Deviation Type Receiver Path

Deviation Specific Additional course changes

Standard Success: 52.6%

Deviation Success: 61.6%

Recommendation(s):

-   -   Incorporate additional course changes in wide receiver path.    -   Investigate additional course changes in all receiver routes.

FIG. 5G is a flowchart showing exemplary steps performed in predictingan opponent's behavior based on detecting deviations in dynamic playexecution and correlating these deviations to previous behavior inspecific situations. FIG. 5H is an exemplary diagram showing a standardroute and a deviant route in similar patterns (paths) run by a slotreceiver. Operation of the present system is best understood by viewingFIGS. 5G and 5H in conjunction with one another.

Using a player location data set for a selected group of participants(such as an offensive football squad) of a team of interest, capturedfor the full duration of a particular situation (e.g., an entire play),the path of each individual participant is determined, at step 5105. Thecollection of these individual paths defines a dynamic play execution.

In step 5110, the dynamic play execution established in step 5105 iscompared against a library (stored in database 115) of well knownclasses of standard play executions and accepted variants of thosestandard executions, for a specific team of interest, to establish abest case match for a selected standard type of play. Once a best casematch has been made, deviations between each path of interest within thedynamic play execution set and the paths defined in the standard libraryversion of that play execution are evaluated, at step 5115.

In FIG. 5H, two paths for offensive player X1 (e.g., a slot receiver) information 5101 are shown—path 5103 is the path selected from the dynamicplay execution established in step 5105, and path 5102 is the path withthe best case match selected from the standard library of plays. Notethat ‘standard’ path 5102 and deviant path 5103 have respective ‘cutpoint’ distances 5111 and 5112. The deviations determined by theevaluation made in step 5115 may be as subtle as a slot receiver cuttinghis ‘in motion’ path short (at arrow 5107) relative to where he wouldnormally change direction at the standard cut point (at arrow 5108), asshown in FIG. 5H.

Having identified a deviation between a path within a dynamic playexecution and the standard library path for that play execution and teamof interest, at step 5120 this deviation is logged to database 115 alongwith a number of associated attributes such as deviation type (e.g.,slot receiver path), deviation specifics (e.g., motion duration),matched library formation (class & variant), situation (e.g., downnumber and yards to first down), and subsequent type of play run.

At step 5125, information in database 115 indicating previousperformances for the team of interest is accessed to retrieve selectedplays for specific situational categories. At step 5130, the plays arethen sorted into groups based on standard play executions for the teamof interest, and the corresponding frequency with which specificbehaviors (e.g., which player ran the ball) occurred are tabulated. Thesorted results are refined based on common, identifiable, and oftensubtle, deviations from the standard play execution. The percentages oftimes specific behaviors occurred (e.g., who the ball was thrown to in aspecific situation) are tabulated for instances when a play executiondeviation was present.

At step 5135, the system accesses information in database 115 toidentify deviations for which there are multiple instances and comparesthe behavior (the specific type of play executed) in specific playexecutions when a particular deviation is present, to behavior when thedeviation is not present.

A defensive squad may want to analyze an offensive squad for specific‘down and distance’ situations on a statistical basis to determine whatan offensive squad typically does when faced with a third down andbetween 7 and 10 yards to first down. Dynamic play deviation informationcan be used to refine a team's prediction ability and improve theirsuccess percentage.

A report is thus generated, at step 5140, to catalog the predictedbehavior of a team of interest as a function of deviant play executionand situations of interest, as determined above. A coaching staff mayuse this report in preparing their team to more accurately predict whatthe team of interest will do in a given situation during a specific playexecution, and how specific deviations in that execution indicate thelikelihood of a particular behavior (e.g., who the ball is thrown to). Atypical report may include the following information:

Behavior Prediction Based on Situation, Dynamic Play Execution andDeviation

Squad: offense

Down: third

Yardage: 7≦x≦10

Formation class: passing

Pass to slot 25%

Pass to other 55%

Run 20%

Formation Variant: slot receiver motion

Deviation Type: shortened motion duration

Pass to slot: 80%

Pass to other: 15%

Run: 5%

In one embodiment, player movements can be traced in real time onto livefeed 109, statically positioned on the field surface as the cameramoves, from detected start of play until detected end of play. Inanother embodiment, player paths are automatically shown in real time ona graphic screen. Data collected (e.g., via feed 107, or from database115) by system 100 is associated with the corresponding video footage;therefore, if a video is selected for replay, the associated data may beused to generate graphic and statistics for combining with, oroverlaying onto, video feed 109.

A generated graphic of the field and players can be a perspective viewwhich allows fading between live action footage and graphic views. Ifthe graphics are generated to have the same aspect ratio and viewingangle as the camera view, player traces and marked paths remain constantwhen fading from generated graphic to camera view. This avoids theswitching from a side perspective view of a camera to a generated planview to show a play. Once transitioned to the generated perspectivegraphic view, the graphic can be rotated to provide the most appropriateviewing angle for showing the play.

FIG. 6A is a flowchart showing exemplary steps performed in usingpositional information to provide real-time automatic annotation of avideo feed 120 of an event. FIG. 6B is an exemplary diagram showingplayer identification graphics and the traced path of a player on avideo display 130. A graphic showing the path of travel of one or moreselected players 660, 661 can be displayed either in real time, or afterthe end of a play. As shown in FIG. 6A, at step 605, trackinginformation from tracking system 108 is received for event participantsof interest. At step 610, the path of travel of one or more of theparticipants is calculated, using positional information calculated fromtracking system 108 data. At step 615, a graphic 652, indicating thepath of travel of the selected participant(s), for example, the path forplayer 661, is overlaid onto the video feed 120, as indicated in FIG.6B.

System 100 can also show, via output 104, the identity and locations ofmultiple players on the field, and their associated teams (e.g., playersof team A in red, players of team B in blue). This information can bedisplayed on a graphic representing the actual playing field, oroverlaid on the live video feed 109, as indicated in step 620.

In one embodiment, the present system keeps continuous track of selectedoff-screen objects so that the off-screen location of the objects isindicated, and the objects are highlighted immediately upon entering thefield of view. A ‘camera view’ coordinate system is used, wherein thecenter of the screen is assigned the coordinate (0,0), the upper lefthas the coordinate (−1, −1), and the lower right is (1,1). Note that theX and Y scales are not the same, since video displays (includingtelevision screens) have an aspect ratio by which the screen width isgreater than the screen height. Thus the point represented by thecoordinate (0.5,0) is located further to the right of center-screen thanthe point represented by coordinate (0,0.5) is located down from thecenter. It should be noted that the coordinate system employed by thepresent system may be different than that described herein and stillprovide the same function.

Using the coordinate system described above, it is relatively simple todetermine if an object is on screen, as both the X and Y coordinateswill be >=−1 and <=1. When reporting the location of an object, itscoordinates can be <−1 or >1, meaning it is off screen. At high zoomlevels, object coordinates can be much larger than 1 or much smallerthan −1.

FIG. 6C is an exemplary diagram showing player identification graphicsindicating the off-screen location of selected objects. By calculatingthe direction of an off-screen object relative to a border of a displayscreen 130, the present system can determine which location along theappropriate screen border is closest to the object. A highlightingindicator or marker 675 is placed at this location proximate the borderof the screen 130 to indicate that the off-screen object (e.g., player676 or 677) is in a particular direction relative to the imagesdisplayed on the screen. Dotted box 670 represents the potential fieldof view of a camera (e.g., video camera 117) which is providing thevideo feed 109 displayed on screen 130. When a previously off-screenobject again becomes visible ‘on-screen’, the marker may change itsappearance and continue tracking the object, as shown in FIG. 6B.

One example of off-screen tracking is a close-up of the quarterback andthe linemen around him (indicated by arrow 671), where two widereceivers 676, 677 are not in view on screen 130, as shown in FIG. 6C.Each wide receiver's general location is indicated with a marker 675(1),675(2) positioned next to the appropriate edge of the screen 130, thusallowing a viewer to tell which wide receiver the quarterback is lookingtoward at a given point in time. Marker 675 may include identifyingtext, or may simply be color-coded to represent one or more players of aspecific type.

Player identities can be indicated via output 104 in real time, forexample, via a player identification graphic 657 overlaid onto the videofeed such that it is close to the player's head or body. Graphic 657shows, for example, the player's number and name, but may, alternativelyor additionally, provide other information such as the number of yardsgained or lost on a particular play, as indicated by graphic 658. Inother embodiments, all of, or certain parts of, selected players 655 maybe highlighted, as indicated by an optionally blinking ‘dot’ 656, orother part of the player, such as the player's head or helmet 655. Oneor more players to be highlighted can be user-selected (via user input118, such as a handheld device described below) or selected by thesystem. For example, the system may automatically identify aquarterback, or all eligible receivers may be automatically identifiedafter a quarterback throws the ball.

In another embodiment, certain players can be highlighted as a result ofdetection of a target situation, such as when two players are within apredetermined distance of each other, e.g., when a receiver is within apredetermined distance of a defensive back.

System 100 can also draw the line of scrimmage and yard markers andoverlay them onto video feed 109. In the case of American football, theapproximate line of scrimmage can be determined from the players' (e.g.,linemens') positions and the distance to a first down can beautomatically calculated and added as an annotation. Participantparameters, such as distance traveled, velocity, and/or acceleration,can also be displayed on a graphic 658 via output 104.

The graphics generated by the present system may be partiallytransparent or opaque, depending on the particular graphic beingdisplayed and whether the graphic is an overlay or not. Graphics mayfade between an image of an event (e.g., live action footage) in thevideo feed and a particular graphic. Graphics may include images thatrepresent actual players, as commonly done in video games.

Graphics may have same aspect ratio and viewing angle as image of anevent, such that player path traces and marked paths remain constantwhen fading between the graphic and the image, thereby providing asmooth transition during the fading process. Alternately, a graphic mayhave a different aspect ratio and/or viewing angle than thecorresponding image to present a view of the event that is differentthan the video image of the event.

FIG. 6D is an exemplary diagram showing a window 685 containing ahighlighting shape 680 overlaid onto a video feed 690. In oneembodiment, rather than modifying the incoming video feed 109frame-by-frame, the present system instead uses a standard video playerto overlay, on top of a video stream 690 (e.g., video feed 109), awindow 685 which includes a ratiometrically correct highlighted image ofeach player being highlighted. This overlay window 685 is transparent,except for semi-transparent areas filled with any color except black. Tocreate a highlight, a white, semi-transparent oval (or a highlightingindicator of other desired color/shape) 680 approximately the size ofthe player to be highlighted (player 681 in FIG. 6D) is drawn on theoverlay window 685 at the approximate position of the player. Theposition of the player is determined from location information extractedfrom tracking system 108. The highlighting indicator 680 is overlaid onthe streamed video image 690 to create an image with highlight 682,while the rest of the video image remains unchanged. With this method,rather than having to deal with the higher bandwidth video data stream,the present system has a simpler and less time-constrained task ofcreating overlay updates independent of more frequent video frameupdates, since the highlighting indicator 680 is re-drawn only when theposition of the highlighted player changes, in the composite displayedimage, by a predetermined displacement.

When a single player is being tracked by the camera, the systemconstantly modifies the zoom level in an effort to maintain thedisplayed player at a relatively constant size in the video frameregardless of how near or far away the player is from the camera. In thecase where only one player is tracked, the sizing of the highlight isrelatively constant except at the minimum and maximum zoom levels.

When other players that are not being tracked appear in the video feed,the highlight size becomes accordingly dynamic. The size of the playerin the video frame, and therefore the required size of the highlight, isgenerally based on how much closer or further away from the camera theseother players are in comparison to the tracked player. In either case(both camera-tracked, and non-camera-tracked players), the systemcontinuously calculates a size-in-the-video-frame metric each time newlocation information arrives for a player. This metric is used todetermine the size of the highlighting shape, and is based oninformation including the video camera location, the location of theplayer(s), the pan & tilt settings of the camera, and the current camerazoom level.

The translation of this information into a size-in-the-video-framemetric involves a series of calculations/transforms includingdetermining a camera's field of view based on pan, tilt and zoom of aplane parallel to the lens, and correcting that field-of-viewmeasurement based on the degree to which the field is not parallel tothe lens (i.e., correcting for camera angle, relative to field). Oncethe field-of-view of the camera (e.g., camera 117) is calculated, thenthe position and size within that field of view is calculated for eachof the location units (on players of interest) within the view. Thiscalculation also corrects for the camera angle. Rather than use the rawnoisy location data, both the field-of-view and thesize-in-the-video-frame calculations are based on filtered locationdata. The filtering may be identical to that used in controlling thecamera motion.

In one embodiment of the present system 100, the path of travel of aparticipant is automatically analyzed and displayed to evaluate theperformance of a participant. FIG. 7 is a flowchart showing exemplarysteps performed in evaluating a participant's performance. As shown inFIG. 7, at step 705, the path of travel of one or more selectedparticipants is determined. The distance traveled by the participant,and/or the participant's velocity may also be determined. At step 710,paths of travel for multiple players are compared to determine how wella particular player was able to perform during a given play (e.g., inavoiding players from an opposing team, or in ‘covering’ anotherplayer). In the case of officials, their paths show where the officialstraveled during a particular play. This information may be helpful inevaluating an official's effectiveness.

At step 715, one or more players whose path meets predetermined criteriais automatically highlighted on a graphic. For example, ‘open’ players(i.e., offensive players who are separated from all defensive players bya certain distance) or blocked players (i.e., those whose velocityduring a certain time period is less than a minimum threshold and whoare positioned in sufficiently close proximity to a player on theopposite team), by changing the color of these players as displayed on agraphic, which may also show the players' path of travel.

A graphic showing a path of travel may also show orientation of theparticipant(s), for example, the direction in which a quarterback orreferee was facing. A graphic may automatically change configuration inresponse to a target situation, for example, a dashed line may bedisplayed during play and a solid line displayed at the end of play.

In one embodiment, system 100 may control the imaging of an event atleast partially in response to event characterization information 104.The system may automatically direct a robotic camera 117 to capture or‘cover’ a target situation such as the beginning of a play, or whencertain players are positioned within a predetermined distance of eachother. For example, a camera may be automatically directed to cover anarea of interest such as the line of scrimmage, a huddle, or aparticular participant or participants in response to, or inanticipation of, a target situation, e.g., camera 117 may be directed tocover a quarterback upon detection of the beginning of a play. Thisprocedure may help ensure that play is not missed due to other action onthe field.

In any game there are a number of situations which have the potential toevolve into a target situation. Some examples include:

Two hockey players who have had confrontations in the past are likely toget into a fight at some point during a game. Every time they are neareach other, they can be targeted in high zoom with a robotic camera 117in anticipation of a target situation.

Two football players have a historically notorious match-up. Every timethey are near each other, they can be targeted in high zoom with arobotic camera 117 in anticipation of a target situation.

A particular basketball player is a good three point shooter. Every timehe is near the three point line, he can be targeted in high zoom with arobotic camera 117 in anticipation of a target situation.

In one embodiment, system 100 has access to the positions of all playerson the field of play and a predefined, prioritized list of conditions towatch for. When system 100 identifies the conditions which precede atarget situation, the system directs a robotic camera 117 to zoom in onand track the appropriate subject(s).

A simple example is a hockey game in which there are two players whofought in the last game. The odds that they will fight again are highand thus any time they are in close proximity, a situation of interestis determined to exist. Should a target situation subsequently occur,high zoom video footage becomes available before occurrence of the eventdefined by the target situation. In the case of a hockey fight there isoften an extended period of close proximity during which glances,gestures and stares are exchanged in a period preceding the actualconfrontation.

System 100 can cause video to be generated only during time intervalswhere the system has detected a play in process. Video buffers maycapture leaders or trailers to ensure that an entire play is recorded.Alternatively, the entirety of an event, or a significant portion of itmay be recorded, in which case the system may automatically post-editthe video recording to remove footage that does not include plays inprogress.

In one embodiment, an entire game video is completely automated. Thisautomation emulates what is typically done with a manually operatedcamera. FIG. 8 is a flowchart showing exemplary steps performed inautomating the video filming of predetermined types of segments of agame. The exemplary process shown in FIG. 8 may be performed for a gameof American football with two cameras, for example, with one camera 117at an end zone and another camera 117 in a press box.

In one embodiment, a camera 117 in the press box (or other vantagepoint) automatically captures all (22) players on the field during theentire game, at step 805. The press box camera may either record theentire field of play or, alternatively, zoom in to more closely capturea smaller area on the field in which all of the players are located. Asindicated in FIG. 8, at step 810, a camera 117 in the end zone is zoomedin on a scoreboard and records for a predetermined duration (e.g., 10seconds). At step 815, upon detection of a predetermined type ofsituation (e.g., players moving to a line of scrimmage), the end zonecamera moves and/or zooms to capture all players on the field. At step820, upon detection of a line set condition, both press box and end zonecameras begin recording. At step 825, upon end-of-play detection, bothcameras continue recording for a predetermined time, e.g., 5 seconds,and then stop recording. If it is not yet the end of the game (step830), then steps 810-825 are repeated until the game ends.

In certain embodiments, system 100 automatically transmits eventcharacterization information to a recipient's wireless device, such as amobile phone, net-book computer, or other portable wireless device,using UDP protocol, for example. FIG. 9 is a diagram showing the use ofa wireless, typically handheld, device 128 with the present system.Users may select which event characterization information and/orvideo/still images of an event are displayed on their mobile device 128.For example, a coach or a spectator may elect to view selected athleteperformance parameters, or a spectator may select one of a number ofvideo feeds, such as one that is covering the spectator's favoriteathlete.

In one embodiment, user-configurable video feeds from one or morecameras 117 at an event facility may be broadcast throughout thefacility. Users with handheld devices 128 may access the specific videofeed of their choice via a wireless broadcast from system 100 or via awireless communication system connected thereto. Coaches, referees,spectators, and commentators may also use handheld devices 128 to choosetheir own particular video feed from system 100.

Coaches and/or officials may also direct event characterizationinformation and/or images to be displayed for training and/or reviewingpurposes on a large display device 130, such as a stadium scoreboard. Acoach may control video playback from the field using a handheld device128 (such as a net-book or other portable computing device) and mayselect video or graphic displays for viewing during training sessions.Replays can be displayed on the handheld device, or on a larger displayunit such as the stadium scoreboard 130.

In American football, a minimum of one referee is assigned to count theplayers on each team just prior to the snap of the ball. This is notonly a difficult task to perform correctly, given the time constraints,it also deters this referee from watching other things immediately priorto the snap.

In one embodiment, system 100 continuously monitors the number ofplayers on each team and notifies referees via handheld devices 128 (viaa tone, vibrating mechanism, etc.) when either team has too many playerson the field at the snap of the ball. The present method also providescoaches with real-time access to players on the field as well as withspecific statistical data regarding their performance. Event spectatorsusing their own handheld devices, capable cell phones, etc., areprovided access to a menu of data options that display information suchas such as who is on the field, statistics, replays, and so forth.

Changes may be made in the above methods and systems without departingfrom the scope thereof. It should thus be noted that the mattercontained in the above description and shown in the accompanyingdrawings should be interpreted as illustrative and not in a limitingsense. The following claims are intended to cover generic and specificfeatures described herein, as well as all statements of the scope of thepresent method and system, which, as a matter of language, might be saidto fall therebetween.

1. A computer-implemented method for determining a target situation inan event comprising: receiving positional information including therelative positions of a first plurality of selected participants in theevent; detecting, in real-time, aggregate motion of the selectedparticipants using the positional information; determining that thetarget situation has occurred when at least one change in the aggregatemotion occurs in accordance with a predetermined characteristic during afirst time interval.
 2. The method of claim 1, wherein the step ofdetecting aggregate motion of the first plurality of selectedparticipants comprises determining a respective velocity of each of thefirst plurality of selected participants from corresponding positionalinformation.
 3. The method of claim 1, wherein the target situation isthe start of a play in the event, and wherein the predeterminedcharacteristic includes an increase in the aggregate motion of at leasta first threshold amount during the first time interval.
 4. The methodof claim 3, wherein a second plurality of selected participants are inpredetermined respective positions prior to the first time interval. 5.The method of claim 3, wherein the aggregate motion remains below asecond threshold value for a second time interval prior to the firsttime interval.
 6. The method of claim 3, further comprising determiningan end of the play after detecting the start of the play, when theaggregate motion decreases by at least a second threshold amount duringa second time interval subsequent to the first time interval.
 7. Themethod of claim 6, wherein the aggregate motion remains above a thirdthreshold value during a third time interval between the first andsecond time intervals.
 8. The method of claim 3, further comprisingcontrolling at least one video camera to start generating a video feedof at least part of the event in response to detecting the beginning ofthe play.
 9. The method of claim 8, further comprising controlling thevideo camera to stop generating the video feed of the event in responseto detecting the end of play.
 10. A computer-implemented method fordetecting a play in an event in real time using data extracted from atracking system providing positional information for event participants,the method comprising: identifying a pre-play target set of conditions;identifying a system arm condition; identifying a play start condition;establishing an aggregate motion baseline of the play; and identifyingan end-of-play condition.
 11. A computer-implemented method forpredicting the relative success of player formations in an eventcomprising: determining relative positions of players of interest in astatic formation; comparing the positions in the static formationagainst a library of the standard formations to find a matching standardformation in the library corresponding to the static formation;determining the deviations in the static formation relative to thematching standard formation; logging the deviations and associatedattributes in a deviation database; accessing the deviation database toidentify correlations between each of the deviations and results ofplays using the deviations; comparing results of plays using thedeviations with results of plays using the matching standard formation;and generating a report including the relative success of the deviationsand the corresponding standard formation.
 12. A computer-implementedmethod for predicting the success of a play in an event comprising:determining relative positions of selected players in a staticformation; comparing the positions in the static formation against alibrary of standard formations to find a matching standard formation inthe library corresponding to the static formation; determiningdeviations in the static formation relative to the matching standardformation; logging the deviations and associated attributes in adatabase; accessing play data to retrieve results of previous playsexecuted by the opponent in a specific type of game situation; sortingthe results based on the matching standard formation and frequency ofplays in the specific game situation; tabulating a percentage of timesspecific plays were run from the corresponding standard formation in thespecific type of game situation; comparing outcomes of plays using thedeviations with outcomes of plays using the matching standard formation;and generating a report including the relative success of the playsusing the deviations and matching standard formation.
 13. Acomputer-implemented method for generating recommendations to improvethe performance of a team in an event comprising: determining paths ofeach player in a selected group for the duration of a play in the eventto establish a dynamic play execution; comparing the dynamic playexecution against a library of standard plays to find a matchingstandard play; determining path deviations for each said path in thedynamic play execution, relative to corresponding standard paths in thematching standard play; logging the path deviations and associatedattributes in a database; accessing the database to identifycorrelations between the path deviations and outcomes of matchingstandard plays; comparing outcomes of the plays with path deviations andcorresponding plays using standard paths; and generating a reportincluding relative success of the plays with the deviant paths andcorresponding plays using standard paths.
 14. A computer-implementedmethod for predicting the behavior of a team of interest in an eventcomprising: establishing paths of each player in a selected group, forthe team of interest, for the duration of a play in the event toestablish a dynamic play execution; comparing the dynamic play executionagainst a library of standard plays for the team of interest to find amatching standard play; determining path deviations for each said pathin the dynamic play execution, relative to corresponding standard pathsin the matching standard play; logging the deviations and associatedattributes in a database; accessing the database to identifycorrelations between the path deviations and outcomes of the matchingstandard plays; tabulating a percentage of times specific plays were runby the team of interest in a specific type of game situation when one ofthe deviations was present; comparing the type of play executed in thespecific type of game situation when one of the deviations was present,with the type of play executed when the play was executed without one ofthe deviations; and generating a report including the predicted behaviorof the team of interest as a function of deviant play execution and thespecific type of game situation, for the team of interest.
 15. Acomputer-implemented method for annotating a video feed of an eventcomprising: receiving the video feed of at least part of the event;receiving positional information from a tracking system including theposition of a selected participant in the event; determining the path oftravel of the participant from the positional information; andoverlaying, onto the video feed, graphical information indicating thepath of travel, and graphical information identifying the participant.16. The method of claim 15, wherein the video feed is a live video feed,and the steps of determining and overlaying are performed in real-time.17. The method of claim 16, further comprising: determining a targetparticipant to be tracked in response to identification of the start ofa play; and causing at least one video camera to track the targetparticipant.
 18. The method of claim 16, further comprising analyzingthe path of travel to evaluate performance of the participant.
 19. Themethod of claim 18, wherein the step of analyzing further comprisesdetermining a distance traveled by the participant and displaying thedistance via graphical information overlaid onto the video feed.
 20. Themethod of claim 16, wherein the participant is selected by a user via awireless handheld device.
 21. The method of claim 16, further including:determining the start of a play in the event by identifying a line setcondition and subsequently detecting an increase in aggregate motion, bya first threshold amount, of a plurality of the participants;determining the end of a play by detecting a decrease in the aggregatemotion of the plurality of the participants, by at least a secondthreshold amount; and determining the path of travel of an official inthe event for the time period between the start of the play and the endof the play; and overlaying, onto the video feed, graphical informationindicating the path of travel of the official.
 22. Acomputer-implemented method for displaying player identificationgraphics in an event comprising: receiving a video feed of the event;receiving positional information from a tracking system including thepositions of selected players in the event; calculating the direction ofan off-screen player, not present in the video feed, relative to aborder of a display screen on which the video feed is displayed, usingthe positional information; determining a location along the borderclosest to the off-screen player; and placing a highlighting indicatoron the display screen proximate to the location to indicate that theoff-screen player is in a particular direction relative to the imagesdisplayed on the screen.
 23. The method of claim 22, wherein, when apreviously said off-screen player again becomes visible on-screen, theappearance of the highlighting indicator is changed.
 24. Acomputer-implemented method for annotating a video stream of aparticipant in an event comprising: receiving the video stream includingthe participant; determining the position of the participant frompositional information provided by a tracking system including theposition of the participant; overlaying, on top of the video stream, atransparent window identical in size to an image of the video streamwhen displayed; drawing a semi-transparent highlighting shape,approximately the size of the participant, in the window at theapproximate position of the participant; and overlaying the highlightingindicator on the video stream to create an image wherein the participantis highlighted.
 25. The method of claim 24, further including re-drawingthe highlighting shape only when the position of the participant changesby a predetermined displacement.
 26. The method of claim 24, furtherincluding calculating a metric indicating the size of the participant inthe image of the video stream, each time new location informationarrives for the participants, wherein the metric is used to determinethe size of the highlighting shape, and is based on informationincluding the video camera location, the location of the participant,the pan & tilt settings of a camera originating the video stream, andthe zoom level of the camera.
 27. The method of claim 24, furtherincluding highlighting one of more of the participants whose path meetspredetermined criteria.
 28. A computer-implemented method for automatingvideo filming of a game on a field comprising: (a) capturing, via afirst video camera, for a predetermined duration, an image of ascoreboard displaying information relating to the game; (b) upondetecting a predetermined pre-play condition, capturing, via the firstcamera, all of the players on the field for the length of the subsequentplay; (c) upon detecting a start of play condition, capturing, via thefirst camera and a second video camera, all of the players on the fielduntil an end of play condition is detected; and repeating steps (a)-(c)until the end of the game.
 29. A computer-implemented method forproviding information related to an athletic event comprising: receivingrespective positions of a plurality of participants on a playing fieldfor the event; analyzing the positions to determine a predefinedparameter in the athletic event; and wirelessly transmitting theparameter to a portable device.
 30. The method of claim 29, wherein oneof the participants is selected by a user via a wireless handhelddevice.
 31. The method of claim 29, wherein the predefined parameter isselected by a user via a wireless handheld device.
 32. The method ofclaim 29, wherein the predefined parameter comprises the participantspresently on the field.
 33. The method of claim 30, wherein thepredefined parameter comprises a correct number of participants in aplaying area, and wherein the portable device automatically provides anindication when an incorrect number of participants are in the playingarea.
 34. A computer-implemented system for displaying athletic eventinformation comprising: a video camera for capturing images of the eventand placing the images in a video feed; a tracking system for providingthe positions of selected participants in the event; a computing device,coupled to the camera, for processing the images from the video feed andthe tracking system to provide game information related to the event; ahandheld device wirelessly coupled to the computing device fordisplaying at least some of the game information, and for issuingcommands to the computing device; and a display device, coupled to thecomputing device; wherein the commands issued from the handheld devicedetermines what aspects of the information are displayed on the displaydevice.
 35. The method of claim 34, wherein the display device is ascoreboard used at the event.
 36. The method of claim 34, wherein thedisplay device is used to display replays of plays in the event selectedvia the handheld device.
 37. A computer-implemented method forcontrolling imaging of an event in which players are participating on afield of play comprising: automatically capturing a target situation bydirecting a robotic camera to cover an area of interest on the field ofplay in anticipation of a target situation and for a period of timeafter the target situation has occurred; wherein the target situation isanticipated at least partially in response to event characterizationinformation including at least one of identification of a playerformation, identification of a play, beginning of a play, end of a play,a player's path of travel, a position of the line of scrimmage, andposition and orientation of coaches and officials.
 38. The method ofclaim 37, wherein said target situation includes the beginning of aplay.
 39. The method of claim 37, wherein said target situation includesan instance in which two selected players are positioned within apredetermined distance of each other.
 40. The method of claim 37,wherein the area of interest is one of a huddle, a selected participant,an area including plurality of selected participants, and the line ofscrimmage.
 41. The method of claim 37, wherein the robotic camera isdirected to track at least one of the players after detection of thetarget situation.
 42. The method of claim 37, targeting an area ofinterest with a robotic camera in anticipation of said target situationwhen one of the players is in predetermined proximity to the area ofinterest.